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STATUS OF CLAIMS 
Claims 1-91 are pending in the present application. Claims 1-12, 29, 34, 36-56, 73, 78, 80- 
87, 90 and 91 stand rejected under 35 U.S.C. § 102(e). Claims 13-28, 30-33, 35, 57-72, 74-77, 79, 
88 and 89 stand rejected under 35 U.S.C. §103(a). Claims 1-91 are appealed. 

STATUS OF AMENDMENTS 
There have been no amendments filed subsequent to the final rejection. 

SUMMARY OF CLAIMED SUBJECT MATTER 
The invention defines a new application programming paradigm (Specification at page 6, line 
26, through page 7, line 6). Existing application authoring approaches have adopted the concept of 
separating the content based aspects of an application from the presentation based aspects 
(Specification page 2, line 2, through page 6, line 2). 

In accordance with the present invention, a new paradigm is introduced, illustratively 
embodied as a Conversational Markup Language (CML), which provides for separating application 
programming into content aspects, presentation aspects and interaction aspects. By focusing on the 
interaction aspect of an application with respect to a user, an application may be written in a manner 
which is independent of the content/application logic and presentation. The "interaction-based 
programming components" recited in independent claims 1, 44, 90 and 91 provide such advantages 
since, as expressly recited, they are independent of content/application logic and presentation 
requirements associated with the application and they may be transcoded on a component by 
component basis to generate one or more modality-specific renderings of the application renderable 
in accordance with one or more modality-specific browsers associated with the one or more 
computer-based devices, such that the interaction-based programming components are independent 
of any modality and any modality-specific browser. Examples of such presentation requirements and 
modality-specific renderings include visual-based (e.g., text and graphical) renderings, speech based 
renderings, and combinations thereof 
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Thus, in accordance with the invention, a device operating with downloaded CML code can 
transcode to, for example, HTML and VoiceXML, substantially simultaneously so as to synchronize 
the multiple browsers providing the user with access to information. Such advantageous 
synchronization according to the invention is possible because the transcoding is done gesture by 
gesture with gesture identification. Thus, when an input/output event occurs in one modality, the 
browser knows what event occurred for what gesture and can immediately update all the supported 
modalities. This results in a very tight synchronization across modalities. Such synchronization is 
also achieved due to the fact that the various modality-specific user interface dialogues, e.g., 
associated with a graphical user interface (GUI) browser or a speech browser, are generated from a 
single CML representation, on a gesture by gesture basis. Thus, the multiple user interfaces, e.g., 
GUI, speech, etc., are synchronized and continuously updated as a user interactively proceeds with 
one or the other modality (Specification, page 7, line 17, through page 8, line 14). 

GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

(1) Claims 1-12, 29, 34, 36-56, 73, 78, 80-87, 90 and 91 are rejected under 35 U.S.C. § 102(e) 
as being anticipated by U.S. Patent No. 6,088,675 to MacKenty et al. (hereinafter "MacKenty"). 

(2) Claims 5 and 49 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
MacKenty in view of "New VXML Forum," posted at Cover Pages Hosted by Oasis (hereinafter 
"New VXML Forum"). 

(3) Claims 13-28 and 57-72 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
MacKenty in view of U.S. Patent No. 6,269,336 to Ladd et al. (hereinafter "Ladd"). 

(4) Claims 31,32, 75 and 76 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
MacKenty in view of U.S. Patent No. 6,569,207 to Sundarsesan (hereinafter "Sundarsesan"). 

(5) Claims 30 and 74 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
MacKenty in view of World Wide Web Consortium document entitled "Extensible Stylesheet 
Language (XSL) version LO" (hereinafter referred to as "W3C XSL specification"). 

(6) Claims 33, 77, and 88 are rejected under 35 U.S.C. §103(a) as being unpatentable over 
MacKenty . in. view.of U.S. Patent No. 6,493,758 to McLain (hereinafter "McLain"). 
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(7) Claims 35, 79, and 89 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
MacKenty in view of U.S. Patent No. 6,456,974 to Baker et al. (hereinafter "Baker"). 

ARGUMENT 

(1) Claims 1-12,29, 34, 36-56, 73, 78, 80-87, 90and91 are rejected under35 U.S.C. §102(e) 
as being anticipated by U.S. Patent No. 6,088,675 to MacKenty et al. (hereinafter "MacKenty"). 

Regarding the §102(e) rejections of claims 1-12, 29, 34, 36-56, 73, 78, 80-87, 90 and 91, 
Appellants assert that MacKenty fails to teach or suggest all of the limitations in said claims for at 
least the reasons presented below. Appellants will reiterate their remarks from their previous 
response dated August 9, 2004 (Section A below), followed by remarks addressing the section in the 
final Office Acfion entitled "Response to Arguments" (Section B below). 

A. It is well-established law that a claim is anticipated only if each and every element as set 
forth in the claim is found, either expressly or inherently described, in a single prior art reference. 
Verdegaal Bros. v. Union Oil Co, of California, 814 F.2d 628, 631, 2 U.S.P.Q.2d 1051, 1053 (Fed. 
Cir. 1987). Appellants assert that the rejection based on MacKenty does not meet this basic legal 
requirement, as will be explained below. 

The present invention, for example, as recited in amended independent claim 1 , recites a 
method of generating an application accessible by a user through one or more computer-based 
devices, comprising the following steps. Interactions that the user is permitted to have with the one 
or more computer-based devices used to access the application are represented by interaction-based 
programming components. The interaction-based programming components are independent of 
content/application logic and presentation requirements associated with the application. Further, the 
interaction-based programming components may be transcoded on a component by component basis 
to generate one or more modalitv-specific renderings of the application renderable in accordance 
with one or more modality-specific browsers associated with the one or more computer-based 
devices, such that the interaction-based programming components are independent of anv modalitv 
and anv modalitv-specific browser . The application is then authored using at least a portion of the 
interaction-based programming components. Independent claim 91 recites similar limitations in an 
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article of manufacture format. Independent claims 44 and 90 recite respective apparatus for 
accessing applications having similar limitations. 

Accordingly, as explained above in the Summary section, a new paradigm is introduced, 
illustratively embodied as a Conversational Markup Language (CML), which provides for separating 
application programming into content aspects, presentation aspects and interaction aspects. By 
focusing on the interaction aspect of an application with respect to a user, an application may be 
written in a manner which is independent of the content/application logic and presentation. The 
"interaction-based programming components" recited in independent claims 1 , 44, 90 and 9 1 provide 
such advantages since, as expressly recited, they are independent of content/application logic and 
presentation requirements associated with the application and they may be transcoded on a 
component by component basis to generate one or more modality-specific renderings of the 
application renderable in accordance with one or more modality-specific browsers associated with 
the one or more computer-based devices, such that the interaction-based programming components 
are independent of any modality and any modality-specific browser . Examples of such presentation 
requirements and modality-specific renderings include visual-based (e.g., text and graphical) 
renderings, speech based renderings, and combinations thereof 

MacKenty is significantly different than the claimed invention. MacKenty discloses at 
column 2, lines 10-17: 

The present invention works with a browser utility, that is, an application for visually 
displaying SGML documents, to present SGML documents to computer users auditorially, 
instead of visually. It parses SGML documents, associates the markup and content with 
various elements of an auditory display, and uses a combination of machine-generated speech 
and non-speech sounds to represent the documents auditorially to a user. Synthetic speech 
is used to read the text content aloud, and non-speech sounds to represent features of the 
document indicated by the markup. For example, headings, lists, and hypertext links can each 
be represented by distinct non-speech sounds that inform the user that the speech they are 
hearing is part of a header, list or hypertext link, respectively. Thus, an SGML page can be 
read aloud using a speech synthesis device, while embedded SGML tags are simultaneously, 
or substantially simultaneously, displayed auditorially using non-speech sounds to indicate 
the presence of special text. 

Thus, the MacKenty system inputs an SGML document which, as disclosed at column 1, 
lines 1 1-20 of MacKenty, may typically be a Hypertext Markup Language (HTML) document, and 
presents the document to a visually-impaired user auditorially. Therefore, since an application in 
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MacKenty is authored in HTML or some other common SGML format (XML, DOCBOOK, etc.), 
such markup language embodies the conventional programming paradigm employed by existing 
appHcations. However, in accordance with the claimed invention, application programming is 
advantageously separated into content aspects, presentation aspects and interaction aspects . Thus, 
as independent claims 1, 44, 90 and 91 recite, the invention provides for representing interactions 
that the user is permitted to have with the one or more computer-based devices used to access the 
application by interaction-based programming components, wherein the interaction-based 
programming components are independent of content/application logic and presentation 
requirements associated with the application . SGML documents are not comprised of components 
that are independent of content/application logic and presentation requirements associated with the 
application. 

Further, since an application in MacKenty is authored in HTML or some other SGML format 
(XML, DOCBOOK, etc.), such markup language is not independent of any modality and any 
modality-specific browser, as recited in the claimed invention. The fact that MacKenty outputs an 
SGML document auditorially in no way anticipates the above-cited claim language. In fact, such 
an operation further strengthens the argument that MacKenty fails to disclose the claimed feature of 
transcoding components on a component by component basis to generate one or more modality- 
specific renderings of the application renderable in accordance with one or more modality-specific 
browsers associated with the one or more computer-based devices, such that the interaction-based 
programming components are independent of anv modality and any modality-specific browser . 

Still further, an SGML document contains elements that are intended to be rendered in only 
one modality, e.g., HTML elements are visually rendered. The fact that the SGML document is 
processed by a "sonification engine" in order to present the document auditorially to a visually- 
impaired user clearly does not teach or suggest the claimed step of representing interactions that the 
user is permitted to have with one or more computer-based devices used to access the application 
by interaction-based programming components, wherein the interaction-based programming 
components are independent of content/application logic and presentation requirements associated 
with the application. That is, while the SGML document is- subsequently presented auditorially 
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rather than visually, the document is still authored in a manner that is intended to be rendered 
visually and, therefore, unlike the claimed invention, is in fact dependent on content/application logic 
and presentation requirements. 

B. Turning now to the "Response to Arguments" section of the final Office Action, the 
Examiner states (on page 4) that "[tjhough HTML may only be intended to be rendered visually, 
MacKenty shows that it can be rendered sonically too, thus making it independent of any modality." 
First, to reiterate, independent claim 1 recites a method of generating an application accessible by 
a user through one or more computer-based devices, comprising the steps of representing interactions 
that the user is permitted to have with the one or more computer-based devices used to access the 
application by interaction-based programming components, wherein the interaction-based 
programming components are independent of content/application logic and presentation 
requirements associated with the application, and further wherein the interaction-based programming 
components may be transcoded on a component by component basis to generate one or more 
modality-specific renderings of the application renderable in accordance with one or more modality- 
specific browsers associated with the one or more computer-based devices, such that the interaction- 
based programming components are independent of any modality and any modality-specific browser, 
and authoring the application using at least a portion of the interaction-based programming 
components. 

That is, as claimed, the application is authored using interacfion-based programming 
components which are independent of content/application logic and presentation requirements 
associated with the application . . . and independent of any modality and any modality-specific 
browser. Thus, the application itself is independent of content/application logic and presentation 
requirements associated with the application . . . and independent of any modality and any modality- 
specific browser. 

In comparison, an application or document authored in any SGML format (e.g., HTML) is 
not itself independent of content/application logic and presentation requirements associated with the 
application . . . and independent of any modality and any modality-specific browser. It is authored 
in a manner that is dependent on content/application logic, presentation requirements, and a specific 
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modality, e.g., in the case of HTML, for a visual-modality. Thus, documents disclosed in MacKenty 
are not authored such that they are independent of content/application logic and presentation 
requirements, modality-independent, or modality-specific browser-independent, as the claimed 
invention requires. 

Second, as previously explained, a process that discloses taking a document authored for a 
visual-modality and processing it using a "sonification engine" in order to present the document 
auditorially to a visually-impaired user, such as disclosed in MacKenty, clearly does not teach or 
suggest the claimed step of representing interactions that the user is permitted to have with one or 
more computer-based devices used to access the application by interaction-based programming 
components, wherein the interaction-based programming components are independent of 
content/application logic and presentation requirements associated with the application . . . and 
independent of any modality and any modality-specific browser. It is respectfully asserted that the 
final Office Action seems to be confusing the process of "authoring" and the process of "rendering." 

Furthermore, suggestions are given in the final Office Action "to differentiate the claims from 
the prior art." However, Appellants respectfully assert that they are entitled to at least the scope of 
the pending claims and, thus, the pending claims are already patentably distinct. 

For at least the above reasons, Appellants assert that independent claims 1 , 44, 90 and 9 1 are 
patentable over MacKenty. 

Further, Appellants assert that dependent claims 2-12, 29, 34, 36-43, 45-56, 73, 78 and 80-87 
are patentable over MacKenty not only because they respectively depend from independent claims 
1 and 44, but also because said claims recite patentable subject matter in their own right. 

By way of example only, claims 8 and 52 recite wherein the user interactions are 
declaratively represented by the interaction-based programming components. Claims 9 and 53 recite 
wherein the user interactions are imperatively represented by the interaction-based programming 
components. Claims 10 and 54 recite wherein the user interactions are declaratively and 
imperatively represented by the interaction-based programming components. Claims 1 1 and 55 
recite wherein the interaction-based programming components comprise basic elements associated 
with a dialog that may occur between the user and the one or more computer-based devices. Claims 
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12 and 56 recite wherein the interaction-based programming components comprise complex 
elements, the complex elements being aggregations of two or more of the basic elements associated 
with the dialog that may occur between the user and the one or more computer-based devices. The 
final Office Action cites column 3, lines 49-65, of MacKenty in rejecting the claims. However, 
nowhere can Appellants find any such features disclosed therein. The portions of MacKenty merely 
disclose details of a parser. 

By way of further example only, claims 34 and 78 recite wherein representation by the 
interaction-based programming components supports a natural language understanding environment. 
The final Office Action cites column 4, lines 15-43, of MacKenty in rejecting the claims. However, 
nowhere therein does MacKenty disclose that representation by the interaction-based programming 
components supports a natural language understanding environment. 

Similarly, claims 80 through 87 are respectively rejected in the final Office Action by citation 
to either column 3, lines 49-65, or column 4, lines 15-43, of MacKenty. However, nowhere does 
MacKenty therein disclose the claimed features. 

Accordingly, withdrawal of the § 102(e) rejections is respectfiilly requested. 

(2) Claims 5 and 49 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
MacKenty in view of "New VXML Forum," posted at Cover Pages Hosted by Oasis (hereinafter 
"New VXML Forum"). 

Regarding the § 103(a) rejections of claims 5 and 49 under 35 U.S.C. § 103(a) based on 
combination of MacKenty and the New VXML Forum, Appellants assert that said claims are 
patentable over the combination not only because they respectively depend fi^om independent claims 
1 and 44, but also because said claims recite patentable subject matter in their own right. 

Further, there is a clear lack of motivation to combine the references. The Federal Circuit 
has stated that when patentability turns on the question of obviousness, the obviousness 
determination "must be based on objective evidence of record" and that "this precedent has been 
reinforced in myriad decisions, and cannot be dispensed with." /« re Zee, 277 F. 3d 1338, 1343 (Fed. 
Cir. 2002). Moreover, the Federal Circuit has stated that "conclusory statements" by an examiner 
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fail to adequately address the factual question of motivation, which is material to patentability and 
cannot be resolved "on subjective belief and unknown authority." Id. at 1343-1344. 

In the final Office Action at page 6, the Examiner provides the following statement to prove 
motivation to combine MacKenty and the New VXML Forum, with emphasis supplied: "[i]t would 
have been obvious for one of ordinary skill in the Computer Networking art at the time of the 
invention to combine the teachings of MacKenty regarding a speech applications system with 
VoiceXML because VoiceXML is a form of SGML document ." 

Appellants submit that this statement is based on the type of "subjective belief and unknown 
authority" that the Federal Circuit has indicated provides insufficient support for an obviousness 
rejection. More specifically, the Examiner fails to identify any objective evidence of record which 
supports the proposed combination. 

Accordingly, withdrawal of the § 103(a) rejections is respectfully requested. 

(3) Claims 13-28 and 57-72 are rejected under 35 U.S.C. § 1 03(a) as being unpatentable over 
MacKenty in view of U.S. Patent No. 6,269,336 to Ladd et al. (hereinafter "Ladd"). 

Regarding the § 103(a) rejections of claims 13-28 and 57-72 under 35 U.S.C. § 103(a) based 
on combination of MacKenty and Ladd, Appellants assert that said claims are patentable over the 
combination not only because they respectively depend from independent claims 1 and 44, but also 
because said claims recite patentable subject matter in their own right. 

By way of example, claims 13-28 and 57-72 recite various detailed features of the 
conversational gestures that make up the language of the invention. The final Office Action cites 
code in Ladd at column 12, lines 30-67, in rejecting the majority of the claims. The other claims are 
rejected based on column 18, lines 56-65. However, column 12, lines 30-67, discloses "syntax and 
grammar that the parser unit of the voice browser utilizes to build a tree structure of the markup 
language document." None of this code, nor anything at column 18, lines 56-65, discloses the 
features of the claimed conversational gestures of the invention. 

Further, based on the rationale given by the Examiner at page 7, there is a clear lack of 
motivation to combine the references based on the rationale of In re Lee, as cited above. 
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Accordingly, withdrawal of the § 103(a) rejections is respectfully requested. 

(4) Claims 3 1 , 32, 75 and 76 are rejected under 35 U.S.C. § 1 03(a) as being unpatentable over 
MacKenty in view of U.S. Patent No. 6,569,207 to Sundarsesan (hereinafter "Sundarsesan"). 

Regarding the § 103(a) rejections of claims 3 1, 32, 75 and 76 under 35 U.S.C. § 103(a) based 
on combination of MacKenty and Sundarsesan, Appellants assert that said claims are patentable over 
the combination not only because they respectively depend from independent claims 1 and 44, but 
also because said claims recite patentable subject matter in their own right. 

Further, based on the rationale given by the Examiner at page 9, there is a clear lack of 
motivation to combine the references based on the rationale of In re Lee, as cited above. 

Accordingly, withdrawal of the § 103(a) rejections is respectftxUy requested. 

(5) Claims 30 and 74 are rejected under 35 U.S.C. §103(a) as being unpatentable over 
MacKenty in view of World Wide Web Consortium document entitled "Extensible Stylesheet 
Language (XSL) version 1.0" (hereinafter referred to as "W3C XSL specification"). 

Regarding the §103(a) rejections of claims 30 and 74 under 35 U.S.C. §103(a) based on 
combination of MacKenty and W3C XSL specification, Appellants assert that said claims are 
patentable over the combination not only because they respectively depend fi'om independent claims 
1 and 44, but also because said claims recite patentable subject matter in their own right. 

Further, based on the rationale given by the Examiner at page 10, there is a clear lack of 
motivation to combine the references based on the rationale of In re Lee, as cited above. 

Accordingly, withdrawal of the § 103(a) rejections is respectfiilly requested. 

(6) Claims 33, 77 and 88 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
MacKenty in view of U.S. Patent No. 6,493,758 to McLain (hereinafter "McLain"). 

Regarding the § 103(a) rejections of claims 33, 77 and 88 under 35 U.S.C. § 103(a) based on 
combination of MacKenty and McLain, Appellants assert that said claims are patentable over the 
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combination not only because they respectively depend from independent claims 1 and 44, but also 
because said claims recite patentable subject matter in their own right. 

Claims 33 and 77 recite wherein representation by the interaction-based programming 
components permits synchronization of the one or more modality-specific renderings of the 
application on the one or more computer-based devices. Claim 88 recites wherein the one or more 
processors are distributed over the one or more computer-based devices and the application is 
synchronized across the one or more computer-based devices. 

The only synchronization that McLain discloses (column 3, line 50, through column 4, line 
6) is that of synchronizing information between a desktop computer 1 6 and a mobile device 1 8. This 
is not unlike "hot-syncing" a personal digital assistant (PDA) with a desktop so that the PDA has the 
up-to-date data from the desktop. This is not the same as representation by the interaction-based 
programming components permitting synchronization of the one or more modality-specific 
renderings of the application on the one or more computer-based devices, as recited in claims 33 and 
77, or the one or more processors being distributed over the one or more computer-based devices and 
the application being synchronized across the one or more computer-based devices, as recited in 
claim 88. 

Further, based on the rationale given by the Examiner at page 10, there is a clear lack of 
motivation to combine the references based on the rationale of In re Lee, as cited above. 
Accordingly, withdrawal of the § 103(a) rejections is respectfully requested. 

(7) Claims 35, 79, and 89 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
MacKenty in view of U.S. Patent No. 6,456,974 to Baker et al. (hereinafter "Baker"). 

Regarding the § 103(a) rejections of claims 35, 79, and 89 under 35 U.S.C. § 103(a) based on 
combination of MacKenty and Baker, Appellants assert that said claims are patentable over the 
combination not only because they respectively depend from independent claims 1 and 44, but also 
because said claims recite patentable subject matter in their own right. 

Claims 35 and 79 recite the step of including code in the authored application for permitting 
cosmetic altering of a presentational feature associated with the one or more modality-specific 
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renderings of the application on the one or more computer-based devices. Claim 89 recites wherein 
the representation of the application further permits cosmetization of the one or more modality- 
specific renderings via one or more modality-specific markup languages. 

The final Office Action cites column 3, lines 7-32, of Baker. However, nowhere therein does 
Baker mention the claimed cosmetization features. The only things mentioned in Baker therein are 
"scroll-up" and "go back" commands. 

Further, based on the rationale given by the Examiner at page 1 1, there is a clear lack of 
motivation to combine the references based on the rationale of In re Lee, as cited above. 

Accordingly, withdrawal of the § 103(a) rejections is respectfully requested. 

In view of the above, Appellants believe that claims 1 -9 1 are in condition for allowance, and 
respectfully request favorable reconsideration. 



Respectfully submitted, 



Date: July 25, 2005 




Attorney for Appellant(s) 
Reg. No. 39,274 
Ryan, Mason & Lewis, LLP 
90 Forest Avenue 
Locust Valley, NY 11560 
(516) 759-2946 
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APPENDIX 

1 . A method of generating an application accessible by a user through one or more computer- 
based devices, the method comprising the steps of: 

representing interactions that the user is permitted to have with the one or more computer- 
based devices used to access the application by interaction-based programming components, wherein 
the interaction-based programming components are independent of content/application logic and 
presentation requirements associated with the application, and further wherein the interaction-based 
programming components may be transcoded on a component by component basis to generate one 
or more modality-specific renderings of the application renderable in accordance with one or more 
modality-specific browsers associated with the one or more computer-based devices, such that the 
interaction-based programming components are independent of any modality and any modality- 
specific browser; and 

authoring the application using at least a portion of the interaction-based programming 
components. 

2. The method of claim 1, in a client/server arrangement wherein at least a portion of the 
application is to be downloaded from a server to at least one of the one or more computer-based 
devices, acting as a client, further comprising the step of including code in the application operative 
to provide a connection to the content/application logic resident at the server. 

3. The method of claim 2, wherein the code in the application operative to provide a 
connection to the content/application logic expresses at least one of one or more data models, 
attribute constraints and validation rules associated with the application. 

4. The method of claim 1, wherein the one or more modality-specific renderings comprise 
a speech-based representation of portions of the application. 
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5. The method of claim 4, wherein the speech-based representation is based on VoiceXML. 

6. The method of claim 1, wherein the one or more modality-specific renderings comprise 
a visual-based representation of portions of the application. 

7. The method of claim 6, wherein the visual-based representation is based on at least one 
of HTML, CHTML and WML. 

8. The method of claim 1 , wherein the user interactions are declaratively represented by the 
interaction-based programming components. 

9. The method of claim 1 , wherein the user interactions are imperatively represented by the 
interaction-based programming components. 

1 0. The method of claim 1 , wherein the user interactions are declaratively and imperatively 
represented by the interaction-based programming components. 

IL The method of claim 1, wherein the interaction-based programming components 
comprise basic elements associated with a dialog that may occur between the user and the one or 
more computer-based devices. 

12. The method of claim 11, wherein the interaction-based programming components 
comprise complex elements, the complex elements being aggregations of two or more of the basic 
elements associated with the dialog that may occur between the user and the one or more computer- 
based devices. 

1 3 . The method of claim 1 , wherein one of the interaction-based programming components 
represent conversational gestures. , _ - - 
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14. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating informational messages to the user. 

15. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating contextual help information. 

16. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating actions to be taken upon successful completion of another gesture. 

17. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating yes or no based questions. 

18. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating dialogues where the user is expected to select from a set of choices. 

19. The method of claim 18, wherein the select gesture comprises a subelement that 
represents the set of choices. 

20. The method of claim 18, wherein the select gesture comprises a subelement that 
represents a test that the selection should pass. 

21. The method of claim 20, wherein the select gesture comprises a subelement that 
represents an error message to be presented if the test fails. 

22. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating rules for validating results of a given conversational gesture. 
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23. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating grammar processing rules. 

24. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating dialogues that help the user navigate through portions of the application. 

25. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating a request for at least one of user login and authentication information. 

26. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating a request for constrained user input. 

27. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating a request for unconstrained user input. 

28. The method of claim 13, wherein the conversational gestures comprise a gesture for 
controlling submission of information. 

29. The method of claim 1, further comprising the step of defining logical input events and 
an association between the logical input events and physical input events that trigger the defined 
logical input events, such that the application may be authored using at least a portion of the 
definitions. 

30. The method of claim 1 , wherein the component by component transcoding is performed 
in accordance with XSL transformation rules. 

3 1 . The method of claim 1 , wherein the component by component transcoding is performed 
in accordance with Java Bean., . 
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32. The method of claim 1 , wherein the component by component transcoding is performed 
in accordance with Java Server Pages. 

33. The method of claim 1, wherein representation by the interaction-based programming 
components permits synchronization of the one or more modality-specific renderings of the 
application on the one or more computer-based devices. 

34. The method of claim 1, wherein representation by the interaction-based programming 
components supports a natural language understanding environment. 

35. The method of claim 1, further comprising the step of including code in the authored 
application for permitting cosmetic altering of a presentational feature associated with the one or 
more modality-specific renderings of the application on the one or more computer-based devices. 

36. The method of claim 1, further comprising the step of including code in the authored 
application for permitting changes to rules for transcoding on a component by component basis to 
generate the one or more modality-specific renderings of the application on the one or more 
computer-based devices. 

37. The method of claim 1 , wherein a definition of an underlying data model being populated 
is separated fi'om a markup language defining the user interaction. 

38. The method of claim 1, wherein a node^id attribute is attached to each component arid 
the attribute is mapped over to various outputs. 

39. The method of claim 1, wherein an author is provided with a pass through mechanism 
to encapsulate modality-specific markup components. 
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40. The method of claim 1, wherein the components may be active in parallel. 

41. The method of claim 1, wherein the representation and transcoding is extensible. 

42. The method of claim 1, wherein a state of the application is encapsulated. 

43. The method of claim 1, wherein the representation permits reference to dynamically 
generated data and supports callback mechanisms to the content/application logic. 

44. Apparatus for use in accessing an application in association with one or more computer- 
based devices, the apparatus comprising: 

one or more processors operative to: (i) obtain the application from an application server, the 
application being programmatically represented by interactions that the user is permitted to have with 
the one or more computer-based devices by interaction-based programming components, wherein 
the interaction-based programming components are independent of content/application logic and 
presentation requirements associated with the application; and (ii) transcode the interaction-based 
programming components on a component by component basis to generate one or more modality- 
specific renderings of the application renderable in accordance with one or more modaility-specific 
browsers associated with the one or more computer-based devices, the interaction-based 
programming components being independent of any modality and any modality-specific browser. 

45. The apparatus of claim 44, wherein the one or more processors are distributed over the 
one or more computer-based devices. 

46. The apparatus of claim 44, in a client/server arrangement wherein at least a portion of 
the application is to be downloaded from a server to at least one of the one or more computer-based 
devices, acting as a client, fiarther comprising the step of including code in the application operative 
to provide a connection to the content/application logic resident at the server. 
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47. The apparatus of claim 46, wherein the code in the application operative to provide a 
connection to the content/application logic expresses at least one of one or more data models, 
attribute constraints and validation rules associated with the application. 

48. The apparatus of claim 44, wherein the one or more modality- specific renderings 
comprise a speech-based representation of portions of the application. 

49. The apparatus of claim 48, wherein the speech-based representation is based on 
VoiceXML. 

50. The apparatus of claim 44, wherein the one or more modality-specific renderings 
comprise a visual-based representation of portions of the application. 

51. The apparatus of claim 50, wherein the visual-based representation is based on at least 
one of HTML, CHTML and WML. 

52. The apparatus of claim 44, wherein the user interactions are declaratively represented 
by the interaction-based programming components. 

53. The apparatus of claim 44, wherein the user interactions are imperatively represented by 
the interaction-based programming components. 

54. The apparatus of claim 44, wherein the user interactions are declaratively and 
imperatively represented by the interaction-based programming components. 

55. The apparatus of claim 44, wherein the interaction-based programming components 
comprise basic elements associated with a dialog that may occur between the user and the one or 
more computer-based devices. . _ 
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56. The apparatus of claim 55, wherein the interaction-based programming components 
comprise complex elements, the complex elements being aggregations of two or more of the basic 
elements associated with the dialog that may occur between the user and the one or more computer- 
based devices. 

57. The apparatus of claim 44, wherein one of the interaction-based programming 
components represent conversational gestures. 

58. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating informational messages to the user. 

59. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating contextual help information. 

60. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating actions to be taken upon successful completion of another gesture. 

61. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating yes or no based questions. 

62. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating dialogues where the user is expected to select from a set of choices. 

63. The apparatus of claim 62, wherein the select gesture comprises a subelement that 
represents the set of choices. 

64. The apparatus of claim 62, wherein the select gesture comprises a subelement that 
represents a test that the selection should pass. _ _ _ 
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65. The apparattis of claim 64, wherein the select gesture comprises a subelement that 
represents an error message to be presented if the test fails. 

66. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating rules for validating results of a given conversational gesture. 

67. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating grammar processing rules. 

68. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating dialogues that help the user navigate through portions of the application. 

69. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating a request for at least one of user login and authentication information. 

70. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating a request for constrained user input. 

71. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating a request for unconstrained user input. 

72. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
controlling submission of information. 

73. The apparatus of claim 44, further comprising the step of defining logical input events 
and an association between the logical input events and physical input events that trigger the defined 
logical input events, such that the application may be authored using at least a portion of the 
definitions. 
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74. The apparatus of claim 44, wherein the component by component transcoding is 
performed in accordance with XSL transformation rules. 

75. The apparatus of claim 44, wherein the component by component transcoding is 
performed in accordance with Java Bean. 

76. The apparatus of claim 44, wherein the component by component transcoding is 
performed in accordance with Java Server Pages. 

77. The apparatus of claim 44, wherein representation by the interaction-based programming 
components permits synchronization of the one or more modality-specific renderings of the 
application on the one or more computer-based devices. 

78. The apparatus of claim 44, wherein representation by the interaction-based programming 
components supports a natural language understanding environment. 

79. The apparatus of claim 44, further comprising the step of including code in the authored 
application for permitting cosmetic altering of a presentational feature associated with the one or 
more modality-specific renderings of the application on the one or more computer-based devices. 

80. The apparatus of claim 44, further comprising the step of including code in the authored 
application for permitting changes to rules for transcoding on a component by component basis to 
generate the one or more modality-specific renderings of the application on the one or more 
computer-based devices. 

81. The apparatus of claim 44, wherein a definition of an underlying data model being 
populated is separated from a markup language defining the user interaction. 



23 



Attorney Docket No. Y0999-478 

82. The apparatus of claim 44, wherein a node_id attribute is attached to each component 
and the attribute is mapped over to various outputs. 

83 . The apparatus of claim 44, wherein an author is provided with a pass through mechanism 
to encapsulate modality-specific markup components. 

84. The apparatus of claim 44, wherein the components may be active in parallel. 

85. The apparatus of claim 44, wherein the representation and transcoding is extensible. 

86. The apparatus of claim 44, wherein a state of the application is encapsulated. 

87. The apparatus of claim 44, wherein the representation permits reference to dynamically 
generated data and supports callback mechanisms to the content/application logic. 

88. The apparatus of claim 44, wherein the one or more processors are distributed over the 
one or more computer-based devices and the application is synchronized across the one or more 
computer-based devices. 

89. The apparatus of claim 44, wherein the representation of the application further permits 
cosmetization of the one or more modality-specific renderings via one or more modality-specific 
markup languages. 

90. A browser apparatus for use in providing access to an application by a user through one 
or more computer-based devices, comprising a machine readable medium containing computer 
executable code which when executed permits the implementation of the steps of: 

obtaining the application from an application server, the application being programmatically 
represented by interactions that the user is permitted to have with the one or more computer-based 
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devices by interaction-based programming components, wherein the interaction-based programming 
components are independent of content/appHcation logic and presentation requirements associated 
with the application; and 

transcoding the interaction-based programming components on a component by component 
basis to generate one or more modality-specific renderings of the application renderable in 
accordance with one or more modaility-specific browsers associated with the one or more computer- 
based devices, the interaction-based programming components being independent of any modality 
and any modality-specific browser. 

91. An article of manufacture for use in generating an application accessible by a user 
through one or more computer-based devices, comprising a machine readable medium containing 
computer executable code which when executed permits the implementation of the steps of: 

representing interactions that the user is permitted to have with the one or more computer- 
based devices used to access the application by interaction-based programming components, wherein 
the interaction-based programming components are independent of content/application logic and 
presentation requirements associated with the application, and further wherein the interaction-based 
programming components may be transcoded on a component by component basis to generate one 
or more modality-specific renderings of the application renderable in accordance with one or more 
modality-specific browsers associated with the one or more computer-based devices, such that the 
interaction-based programming components are independent of any modality and any modality- 
specific browser; and 

authoring the application using at least a portion of the interaction-based programming 
components. 
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